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CLAIMS 

What is claimed is: 

1. A processor-based method of combining valuations of multiple choices 
from each of multiple evaluators, the method comprising: 

for each evaluator, 

sorting the choices into an order that indicates preference; 
determining interval sizes between choice valuations; 
obtaining a rank for each interval size; and 

using interval size ranks, at least in part, to determine an evaluator 
score for each choice; and 
combining evaluator scores for each choice to determine, at least in part, 
an overall score for each choice. 

2. The method of claim 1 , wherein said obtaining a rank for each interval size 
comprises ranking interval sizes from smallest to largest. 

3. The method of claim 1 , wherein said using interval size ranks to determine 
an evaluator score for each choice comprises: 

associating a predetermined score with a first choice; and 
iteratively calculating a score for each subsequent choice by combining the 
score of a preceding choice with an increment based on the rank of 
the interval size between the preceding choice and said subsequent 
choice. 

4. The method of claim 1 , further comprising: 

determining equivalence classes for choices based on choice valuations, 
wherein said determining equivalence classes precedes said 
obtaining a rank for each interval size, and wherein said obtaining a 
rank for each interval size involves obtaining a rank for each interval 
size between valuations associated with different equivalence 
classes; 
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calculating scores for equivalence classes based on said inten/al size 
ranks; and 

associating the score for each equivalence class with each choice that is a 
member of that equivalence class. 

5. The method of claim 4, wherein said determining equivalence classes is 
based on a statistical interval distribution. 

6. The method of claim 4, wherein said determining equivalence classes is 
based on a predetermined minimum interval size. 

7. The method of claim 4, further comprising: generating a valuation to 
associate with each equivalence class by operating on the one or more 
valuations associated with the one or more choices in that equivalence class, said 
operating using an operator from a group consisting of: 

an arithmetic mean of the one or more valuations; 
a geometric mean of the one or more valuations; 
a harmonic mean of the one or more valuations; 
a root mean square of the one or more valuations; 
a median of the one or more valuations; 

a value corresponding to a predetermined percentile among the one or 

more valuations; 
a most frequent valuation of the one or more valuations; 
a most frequent valuation determined within a predetermined precision; 
a maximum of the one or more valuations; and 
a minimum of the one or more valuations. 

8. The method of claim 1 , further comprising: 

assigning an adverse valuation to each choice not evaluated by a given 
evaluator, wherein the adverse valuation indicates that the choice is 
less preferred than the least preferred choice evaluated by the 
given evaluator, and wherein the adverse valuation separates the 
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choice from the least preferred choice by an interval greater than 
each other interval between adjacent choice valuations. 

9. The method of claim 1 , wherein said ranking includes: 
assigning equalized ranks to intervals that are substantially equal. 

10. The method of claim 1, wherein the overall score for each choice is a 
weighted or unweighted sum of evaluator scores for that choice. 

1 1 . The method of claim 1 , wherein said combining includes: 
scaling evaluator scores to a uniform range. 

12. The method of claim 1 , wherein said combining includes: 

scaling evaluator scores to provide each evaluator with an equal sum of 
evaluator scores. 

13. The method of claim 1 , further comprising: 

selecting a predetermined number of choices having the most favorable 
overall scores. 

14. The method of claim 1 , further comprising: 

selecting from a group of choices a predetermined number of choices 
having the most favorable overall scores. 

1 5. The method of claim 1 , further comprising: 

sorting the choices according to the overall score determined for each 
choice. 

16. The method of claim 1, wherein at least one of the evaluators is selected 
from a group consisting of: 

a provider of values derived from sensor measurements, 

a provider of values derived from performance measurements, 
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a provider of values derived from solicited personal opinions, 

a provider of values derived by a Bayesian network, 

a provider of values derived by a decision tree, 

a provider of values derived by an evolved algorithm, 

a provider of values derived by a neural network, 

a provider of values derived by a fitness estimator, and 

a provider of values derived from complexity measurements. 

17. The method of claim 1, wherein each choice represents an alternative 
from a group consisting of: 

classifications of an item; 
candidate solutions to a problem; 
contest competitors; and 
features to be used for classification. 

18. A system for combining valuations of multiple choices from each of 
multiple evaluators, the system comprising: 

a processor; and 

a memory coupled to the processor and configured to provide the 
processor with software that configures the processor to iterate 
through the multiple evaluators, and for each evaluator configures 
the processor to: 

obtain ranks of intervals between a valuation-ordered group of choices; 
and 

determine a score for each choice from the inten/al ranks, 
wherein the software further configures the processor to combine choice 
scores for each evaluator to obtain an overall score for each choice. 

19. The system of claim 18, wherein the software further configures the 
processor to retrieve choice valuations from at least one external evaluator. 
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20. The system of claim 18, wherein the software further configures the 
processor to operate as at least one evaluator. 

21. The system of claim 18, wherein the software further configures the 
processor to employ one or more of the choices in a manner responsive to the 
overall scores. 

22. The system of claim 18. wherein as part of configuring the processor to 
rank intervals, the software configures the processor to assign a rank of zero to 
Intervals of insignificant size. 

23. The system of claim 18, wherein as part of configuring the processor to 
determine a score for each choice, the software configures the processor to: 

determine equivalence classes for choices based on choice valuations 
before obtaining ranks of intervals, wherein said obtaining 
comprises finding ranks of the intervals between valuations 
associated with the equivalence classes; 
calculate scores for equivalence classes based on the interval ranks; and 
associate the score for each equivalence class with each choice that is a 
member of that equivalence class. 

24. The system of claim 18, wherein the software further configures the 
processor to assign an adverse valuation to each of an evaluator's unevaluated 
choices, wherein the adverse valuation indicates that the choice is less preferred 
than the least preferred choice evaluated by that evaluator, and wherein the 
interval between the least preferred choice and the unevaluated choice no 
smaller than each interval between that evaluator's evaluated choices. 

25. The system of claim 1 8, wherein as part of configuring the processor to 
rank intervals, the software configures the processor to assign equalized ranks to 
intervals that are substantially equal. 
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26. The system of claim 18, wherein as part of configuring the processor to 
combine choice scores, the software configures the processor to scale choice 
scores to a uniform range. 

27. The system of claim 18, wherein as part of configuring the processor to 
combine choice scores, the software configures the processor to scale choice 
scores to provide each evaluator with an equal sum of choice scores. 

28. The system of claim 18, wherein the overall scores for each choice are 
weighted or unweighted sums of the scores for each choice. 

29. An evaluation system comprising: 

a means for determining a valuation by each of multiple evaluators for 

each of multiple choices; 
a means for ranking intervals between valuation-ordered choices for each 

evaluator; 

a means for determining choice scores using interval ranks for each 
evaluator; 

a means for combining choice scores to determine an overall score for 
each choice; and 

a means for selecting a predetermined number of choices in accordance 
with their overall scores. 

30. The method of claim 29, wherein said means for determining choice 
scores comprises: 

a means for associating a predetermined score with a first choice; and 
a means for iteratively calculating a score for each subsequent choice by 
combining the score of a preceding choice with an increment based 
on the rank of the interval size between the preceding choice and 
said subsequent choice. 
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31. An information carrier medium configurable to provide a computing device 
with software for combining valuations of multiple choices from each of multiple 
evaluators, wherein the software configures the computing device to: 

for each evaluator, 

sort the choices into an order that indicates preference; 
determine inten/al sizes between choice valuations; 
obtain a rank for each inten/al size; and 

determine from the interval size ranks an evaluator score for each 
choice; and 

combine evaluator scores for each choice to determine an overall score for 
each choice. 

32. The medium of claim 31, wherein as part of configuring the computing 
device to determine an evaluator score for each choice, the software configures 
the computing device to: 

associate a predetermined score with a first choice; and 
iteratively calculate a score for each subsequent choice by combining the 
score of a preceding choice with an increment based on the rank of 
the interval size between the preceding choice and said subsequent 
choice. 

33. The medium of claim 31, wherein the software further configures the 
computing device to: 

determine equivalence classes for choices based on choice valuations 
before obtaining a rank for each interval size, wherein said 
obtaining a rank for each interval size involves obtaining a rank for 
each interval size between valuations associated with different 
equivalence classes; 

calculate scores for equivalence classes based on said interval size ranks; 
and 

associate the score for each equivalence class with each choice that is a 
member of that equivalence class. 
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